Task description

Let us consider a model corresponding to a random sample from a bivariate gaussian distribution \(\mathcal{N}_2 \left( \boldsymbol{\mu}, \, \Sigma\right),\) where \(\boldsymbol{\mu} = (1,2)^T\) and \(\Sigma = \begin{pmatrix} 2 & 1 \\ 1 & 4 \end{pmatrix}\). We would like to investigate the performance of the tests of hypothesis \[ H_0: \mathbb{A}\boldsymbol{\mu} = a \quad \text{against} \quad H_1: \mathbb{A}\boldsymbol{\mu} \neq a,\] where \(\mathbb{A} = (2,-1)\) and \(a = 0\).

First test assumes known variance matrix \(\Sigma\) and uses the statistic \[ T_{n} = n(\mathbb{A}\overline{\boldsymbol{X}}_{n} - a)^\top(\mathbb{A}\Sigma \mathbb{A}^\top)^{-1} (\mathbb{A} \overline{\boldsymbol{X}}_{n} - a)\] which has under the null hypothesis \(\chi^2_1\) distribution. Second test does not assume the known variance matrix \(\Sigma\) and it uses the sample variance matrix \(\mathcal{S}\) and the test statistic \[ \tilde{T}_{n} = (n - 1)(\mathbb{A}\overline{\boldsymbol{X}}_{n} - a)^\top(\mathbb{A}\mathcal{S} \mathbb{A}^\top)^{-1} (\mathbb{A} \overline{\boldsymbol{X}}_{n} - a)\] which has under the null hypothesis \(T^2_{1,n-1}\) distribution.

Tests performances

Let us perform a series of described tests on the level of \(\alpha = 0.05\). Firstly, we will simulate \(k = 1000\) samples of consecutive sizes \(n = 10, 20, 30, 50, 100, 200, 500, 1000\) from the null hypothesis and we will look on the type I error, i.e. the proportion of falsely rejected nulls. The picture below shows that both tests attain the selected level of \(\alpha = 0.05\) even for small sample sizes, i.e. as expected.

Let’s compare the power of the two tests. We will use the same level and sample sizes, but in this case we will generate the samples from an alternative hypothesis, specifically we will shift the vector of means to \(\boldsymbol{\mu} - \boldsymbol{\mu_s}\), where \(\boldsymbol{\mu_s}\) will be sequentially \(\begin{pmatrix} 0 \\ 0.3\end{pmatrix}, \begin{pmatrix} 0 \\ 0.5\end{pmatrix}, \begin{pmatrix} 0 \\ 1\end{pmatrix}, \begin{pmatrix} 0 \\ 2\end{pmatrix}\). In these cases, the values of \(a=\mathbb{A}\boldsymbol{\mu}\) are \(0.3, 0.5, 1, 2\) when the null hypothesis assumes that \(a = 0\). Lets look at the figure below.

The figure suggests that both tests are consistent for all of the alternatives. Both small sample size and small deviance from the null hypothesis weaken the power of the tests. For small sample sizes we can also observe a notable difference in the performance of both tests. Especially for alternatives that differ more from the null, the test that assumes the known variance matrix outperforms the other test that uses an estimated variance matrix. However, this difference is diminishing with an increasing sample size.